Field of the Invention
This invention relates to methods and apparatus for the measurement of human eye movements.
Description of Related Art
The measurement of human eye movements is important for the assessment of damage to the eyes, optic pathways, and brain mechanisms controlling oculomotor behavior. The eyes see clearly in only one central region, so it is necessary for the eyes to direct their gaze directly on any target object of interest in order to see it clearly. In order to look directly at any target, the eyeballs must rotate or move in their sockets so that light from the target is centered on the central region or fovea of the retina. The efficiency of such eye rotations or movements is called oculomotor performance and, for accurate binocular vision in 3D space, the two eyes need to be coordinated to target or look at the same object of interest simultaneously in both eyes, with both lenses focusing on the target. Moreover, effective vision requires the pupil to adjust the light level reaching the retinas for an optimal match to the dynamic range of the retina's photoreceptors. Such coordinated targeting, lens focusing, and light-level adjustment is called binocular coordination and is a primary function of the visual process that is necessary for effective vision.
One valuable use for eye movement measurement is for the diagnosis of brain damage and drug toxicity, which produce a characteristic slowing of eye movement dynamics. Also, another important reason for measuring binocular eye movements is in the diagnosis of strabismus conditions, in which one eye deviates from the target or gaze angle of the other eye, indicating a loss of binocular coordination that requires medical treatment. There are many forms of strabismus with diverse causes that are difficult to distinguish, making the ability to quantify the dynamics of binocular eye movements an important diagnostic tool.
Many systems have been patented for both head-mounted and remote camera eye tracking, typically from one, or both, of two kinds of infrared signals from an eye: corneal reflections and pupil images. These include U.S. Pat. No. 4,973,149 to Hutchinson (1990), U.S. Pat. No. 5,325,133 to Luo (1994), U.S. Pat. No. 5,668,622 to Charbonnier and Masse (1997), U.S. Pat. No. 5,912,721 to Fukui, Okamoto, and Yamaguchi (1997) and the Google Glass monocular eye tracking system described in U.S. Pat. No. 8,398,239 to Horning, Ohnstein, and Fritz (2013).
U.S. Pat. No. 6,568,809 to Trajkovic, Gutta, and Colmenarez (2003) describes a device for automatic adjustment of a zoom lens that incorporates a binocular gaze tracking system but only refers to gaze tracking capabilities that are “known in the art”, without providing any specifics.
U.S. Pat. No. 6,926,429 to Barlow, Harter, Scharenbroch, and Newman (2005) and a paper by Bretzner and Krantz (“Towards low-cost systems for measuring visual cues of driver fatigue and inattention in automotive applications”, Proceedings of the IEEE International Conference on Vehicular Electronics and Safety. 2005, pp. 161-164, 2005) describe monocular eye tracking by means of infrared imaging systems using the iris-corneal boundary (limbus) as the indicator of eye position.
Despite their different methodologies, all these systems require external apparatus close to the eyes, some mounted on the head by means of a spectacle frame, goggles or a headband of some kind, others mounted on the table in front of the viewer, and all require calibration procedures.
Recent publications have described eye-tracking of the iris from video sequences based on pattern recognition of the configuration of features in the eye region of the face with an artificial neural network in a series of calibrated eye fixation positions (Baluja and Pomerleau, “Non-intrusive gaze tracking using artificial neural networks.” Research Paper CMU-CS-94-102, School of Computer Science, Carnegie Mellon University, Pittsburgh Pa., USA, 1994; Holland and Komogortsev, “Eye tracking on unmodified common tablets: Challenges and solutions.” ACM Symposium on Eye Tracking Research & Applications (ETRA 2012), 2012). The latter authors developed this system for tablet computer applications, indicating the broad promise of device-free eye-position tracking capability. Their system had a sampling rate of about 1 frame every 5 seconds, which is sufficient for monitoring eye position at a slow rate, but entirely insufficient for the assessment of oculomotor dynamics requiring frame rates of 30 per second or higher.
A paper by Nagamatsu, Iwamoto, Sugano, Kamahara, Tanaka, and Yamamoto, “Gaze estimation method involving corneal reflection-based modeling of the eye as a general surface of revolution about the optical axis of the eye.” Institute of Electronics, Information and Communication Engineers Transactions, 95-D(6):1656-1667, 2012, describes a video-based gaze estimation system incorporating a 3D model of the eyeballs and irises. They do not include the eyelids or the pupils as components of their model. Their system uses two infrared cameras is designed to estimate gaze angles by averaging over a minimum of 30 frames, and is therefore not suitable for tracking rapid eye movements.
A key issue both for understanding the focus of interest of a person and for diagnosing any deficiencies in their oculomotor capabilities is the capability of tracking the positions of the eyes. Although this capability may be achieved by a variety of specialized equipment, such as infrared pupil trackers mounted on glasses or goggles, such equipment constrains the person's activities in various ways, and may interfere with the procedures of an eye care professional assessing deficiencies of their oculomotor capabilities. Heretofore in order to assess the positions of both eyes through the use of images captured by a distant video camera trained on the face, it was necessary to use specialized eye-tracking equipment, and this had to be calibrated with a series of eye fixations on targets in known positions in space.
Most previous approaches to the capability of tracking gaze angles from a video frame have the one or several of the following limitations:                a) They measure the movements of one eye only.        b) They require prior calibration of the relationship of the configurations of image features to the eye rotation vectors.        c) They do not provide the ability to track to the extreme eye positions needed for quantifying ophthalmic and optometric evaluations of oculomotor control.        d) They are subject to errors in the eye movement estimation due to interference from concurrent head movements        e) They do not provide analysis of the ocular dynamics of the eye rotation.        
As a consequence of these shortcomings, the state of the art in video gaze tracking was recently characterized by Hansen and Ji (“In the eye of the beholder: a survey of models for eyes and gaze.” IEEE Transactions on Pattern Analysis and Machine Intelligence. 32, 478-500, 2010) with the following statement: “Another future direction will be to develop methods that do not require any calibration. This does not seem to be possible given the current eye and gaze models.”